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The following meeting summary was edited by IES staff for clarity and consistency. Panel participants 
were allowed to review and comment on it, and their corrections were incorporated and clarifications 
were included as needed. 
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Advancing Adult Education Research 
Meeting Summary 


April 26, 2012 
8:30 a.m. — 4:30 p.m. 


Goal of the Meeting: The goal of this meeting is to discuss the challenges of adult education 
and mechanisms to better understand and address those challenges through research. The 
meeting was structured around three panel discussions representing the perspectives of State 
directors of adult education programs, adult education researchers, and education research 
methodologists and statisticians. 


Welcome and Introductions: Elizabeth Albro, Ph.D., and Meredith Larson, Ph.D., 
National Center for Education Research (NCER) 

Dr. Albro called the meeting to order at 8:37 a.m. She welcomed the participants, who brought 
their expertise about adult learners, and hoped they would advise NCER on issues around 
conducting high-quality, rigorous research among adult learners outside the university setting. 
The National Research Council recently published Improving Adult Literacy Instruction: Options 
for Practice and Research, which describes the state of adult education and lays out the 
challenges. NCER now has a research portfolio on the topic of postsecondary adult education 
that will provide more funding opportunities for adult education research. 


Dr. Larson pointed out that 90 million Americans are at or below the threshold for basic reading 
skills, and about 3 million people are involved in some kind of adult education each year. She 
hoped the meeting would identify some issues that NCER can help to address with existing 
tools and resources as well as pinpoint some challenges for which new research methods or 
tools are needed. 


State Perspectives 
Anne Serino, Administrator, Office of Adult and Community Learning Services, 
Massachusetts Department of Elementary and Secondary Education 


Ms. Serino noted that State program directors are constantly asked to make the case for why 
adult education is important—even more often than they are asked to describe program 
outcomes. The question can be answered on multiple levels that describe the impact of adult 
education on individuals, communities, States, employers, and others. The “why” of adult 
education is central, said Ms. Serino, and she hopes researchers will keep the question in mind 
as they pursue their work. She provided some examples of research efforts that helped advance 
adult education in Massachusetts. 


Ms. Serino said the National Center for the Study of Adult Learning and Literacy (NCSALL) 
published their reading research along with a study guide that helped practitioners think about 
and use the research findings. Adult education staff are not necessarily well versed in research 
and evaluation, so such tools are helpful. The NCSALL findings also convinced Massachusetts 
to join the Office of Vocational and Adult Education’s (OVAE’s) Student Achievement in Reading 
(STAR) initiative. Thus, one U.S. Department of Education (ED) effort provided research 
findings, a guide to using the findings, and a clear action that States could take to contribute to 
the evidence base and broaden their own efforts. 


To give another example of how State directors use research, Ms. Serino talked about 
Massachusetts’ Even Start Family Literacy Program. This program began in 1989 and showed 


IES NCER Advancing Adult Education Research Meeting 1 


some impact on children but never the “wow” factor that its supporters hoped for, said Ms. 
Serino. Although Even Start is no longer funded, the State looked closely at the most successful 
component of the program—the family action plan, which connected families with adult learning 
opportunities specific to their own goals. Analysis revealed that deliberate, purposeful planning 
with the adult learner yielded good outcomes. Beginning in 2012, advisors in adult community 
learning centers in Massachusetts must help participants develop individualized education and 
career plans that map to learners’ goals. Ms. Serino emphasized that despite Even Start’s 
failure to demonstrate the impact its creators hoped it would have, the State used it to identify 
what did work and tried to replicate that success in other programs. 


Ms. Serino also noted that the family literacy initiative prompted case studies that yielded some 
remarkable results. Parents articulated the transformative nature of the process. For example, 
one young mother sought to improve her literacy so that she could take part in an office training 
program, and she found she was finally able to help her daughter with homework. Stories like 
this resonate among funders and legislators and raise morale among educators. 


Ms. Serino noted that when adult education research is not available, K-12 research can be 
illuminating. Although there is no research on ideal class size for adult basic education (ABE), a 
great deal of K-12 research indicates that class size matters in settings where there are low- 
income students and teachers who are not well prepared. Both of these factors are common in 
adult education settings, so Ms. Serino used the K-12 data to push back against efforts to 
increase ABE class sizes in Massachusetts. 


Summarizing, Ms. Serino said research should continue to answer the question of why adult 
education is important and provide the field with findings that can inform policy decisions. She 
also noted that the best way to win over practitioners is for researchers to explain how their 
findings help students. 


Michael Westover, Director, Bureau of Postsecondary and Adult Education 
Pennsylvania Department of Education 


Mr. Westover said that because he oversees a wide range of activities, he needs research 
findings that are digestible and portable both for policymakers and adult education providers. 
Adult education staff are a part-time, itinerant workforce and they need research they can 
understand and implement immediately. Recognizing that adult education staff are usually not 
well prepared for teaching, the State invested millions in professional development through one 
institution. The program appeared to succeed and grew to a budget of $25 million. Looking 
more closely at the program, it became clear that the money could be better spent by shifting 
from workshop-based offerings to a classroom-based approach—fostering a community of 
peers open to discussion and taking an intensive, long-term approach. The new approach 
provides adult education staff advice from subject-matter experts, opportunities to talk with 
peers who have succeeded in teaching the same topic, classroom observations, and 
consultants who help with follow-up. 


Mr. Westover emphasized that he has a lot of leeway to change and implement policy quickly, 
which he uses to respond to urgent needs. 


Discussion 
Funding 


Ms. Serino said providers compete for funding every 5 years and that the open competition 
process is the best mechanism for making changes. Mr. Westover said that last year, 
Pennsylvania required all of its programs to compete. As a result, the State consolidated 200 
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programs down to 59. Despite a 40 percent reduction in funding, the competition sought 
programs that would provide more case-management, more in-house professional 
development, and more spending per student. The State hoped that part of the outcome of 
these changes would be to attract higher skilled teachers who can better meet the needs of the 
high percentage of adult learners with learning disabilities. State funding competitions often use 
the criteria described in the Workforce Reinvestment Act. Ms. Serino said Massachusetts relies 
heavily on the demonstration of past effectiveness (over the past 5 years, using State data). 


Mr. Westover added that Pennsylvania and Massachusetts both have a lot of federal funding to 
support research and development (e.g., $4 million in Pennsylvania). Other smaller States have 
much less money with which to work. Ms. Serino noted that Massachusetts provides State 
funds equal to three times the federal allocation. 


Professional Development 


Mr. Westover elaborated on the State’s new professional development model. Following 
classroom observation, teachers are encouraged to evaluate their own success, consider what 
tools could be helpful, identify areas for improvement, and determine what tools or education to 
pursue. The approach takes a lot more time than workshop-based development, he said, and 
the State relies on consultants, peers, and administrators to observe, share, and implement 
what works. Teachers resented leaving their classes for workshops. Now, educational 
opportunities are offered online and in the evenings, with consultants coming to the classroom, 
and teachers feel their time is better used. 


Mr. Westover pointed out that adult education teachers’ wages are very low. He would like to 
address the disparity between teachers’ and administrators’ compensation, but local programs 
complain that changing the policy would leave less money for students. 


Data and Research Models 


Individual programs are funded on the basis of the data they collect and report to the National 
Reporting System for Adult Education (NRS). Mr. Westover said that his office is responsible for 
monitoring the accuracy of those data in Pennsylvania; therefore, he would like to have a source 
of unbiased data for comparison. Programs collect individual student-level information, but Mr. 
Westover questioned the accuracy and completeness of the data. Dr. Wrigley added that in the 
national studies she has been involved in, the organization conducting the studies always 
collected their own data instead of relying on program data which was not always reliable. Ms. 
Serino said Massachusetts has a database of 110,000 student records matched with a national 
student clearinghouse going back to 2002. She hoped such data could be mined by skilled 
researchers. Identifying denominators is an ongoing problem, said Ms. Serino. 


Ms. Keenan, director of Division of Adult Education and Literacy, OVAE, clarified that States 
collect individual student data and that those data must meet certain standards. She said that 
State unique identifiers ensure that individuals are counted only once in the data sets. The 
National Reporting System (NRS) for adult education collects aggregate State data that include 
program outcomes such as educational gain which is measured through standardized pre- and 
post-testing. Ms. Keenan clarified that the assessments used by local programs must meet 
specific psychometric criteria and be approved by ED for use in the NRS. 


Mr. Westover noted that in Pennsylvania, staff at Pennsylvania State University review their 
program data and correct them as needed before sending them to the NRS. Furthermore, each 
student has a unique Pennsylvania identification number that helps minimize duplication and 
allows for matching from pre-K through community college or State institution. 
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Pennsylvania student data can be mapped to a teacher’s education level and years of 
experience. Mr. Westover hopes to link teachers’ professional development efforts with student 
records to find correlations between student outcomes and teacher preparedness. Ms. Serino 
noted that Massachusetts collects a lot of data on program design and structure. She believes 
that her State’s programs are more similar than different. 


Dr. Wrigley pointed out that large-scale studies are hindered by the fact that adults do not stay 
in programs long enough to benefit from participation. New models that combine self-access 
and self-directed learning with face-to-face education are needed. In particular, English as a 
second language (ESL) students may benefit from more learning-how-to-learn strategies that 
take advantage of other (non-classroom based) learning opportunities, including online options. 
Clearly, adults need alternatives to programs that require multiple, face-to-face sessions. Dr. 
Gregg agreed on the need to consider how to use new technology to reach students and 
enhance professional development. Mobile phones as well as computer applications should be 
considered. 


Other 


Mr. Westover explained that waiting lists for adult classes are common. However, once a 
potential student is placed on a waiting list, that individual usually does not come back. 


Research Perspectives 
Heide Spruck Wrigley, Ph.D., Literacywork International 


Dr. Wrigley explained that her organization focuses on the education and training of vulnerable 
youth and adults with a particular emphasis on non-native speakers of English. As part of that 
work, the organization addresses immigrant education policies. She noted that policies such as 
the new requirement that students have a GED (general education diploma/general equivalency 
diploma) in order to obtain a Pell grant may be especially harmful to immigrant populations with 
low levels of education and may lead to the loss of social capital. Dr. Wrigley noted that her 
early research on literacy in ESL programs involved case studies that were translated into a 
handbook (Bringing Literacy to Life: Issues and Options in Adult ESL Literacy) to make the 
research more accessible to teachers and administrators in the field. More recently, Dr. Wrigley 
and colleagues conducted a large study (funded by IES) on the impact of a reading intervention 
for low-literacy ESL students and found the intervention had no effect. She felt the disappointing 
results could be attributed to several factors, all of which can inform future research efforts: 


e The definition for students included in the study was very narrow, and programs struggled 
to recruit enough students to meet the large numbers of appropriate subjects needed for an 
experimental study (students with low levels of education in the home country who 
functioned at beginning levels of ESL)' 


e The study did not take into account the wide variations of levels in native-language literacy 
among students, which strongly affect the ability to learn a second language. The stronger 
the literacy skills in the native language, the easier it is to acquire English literacy.” A 


"Dr. Wrigley later noted that “[a] number of case studies with smaller numbers may have been more appropriate for 
this population whose numbers in ABE programs are relatively small, but whose issues (lack of progress) deserve 
attention) 
? Dr. Wrigley later noted that “[t]his is true even if the students in question have literacy in a language that uses the 
non-Roman alphabet. The study included students from non-Roman alphabet languages and did not assess for 
literacy in the native language. In the end, it turned out that a large number of students with fairly high levels of 
literacy in the mother tongue were included, possibly skewing the results.” 


” 
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suggestion to assess levels of native language literacy was rejected. The intervention did 
not include enough teacher training or classroom observations, and fidelity of 
implementation was not strongly monitored. The intervention was not sufficiently distinct 
from what teachers normally do. Anecdotal evidence suggested that several of the teachers 
blended the intervention into their normal practices. 


e The advisory panel did not include enough respected representatives from the adult ESL 
field. The advisors (researchers in reading and one researcher in second language 
acquisition did not have a great deal of experience transferring knowledge from the K-3 
reading research into a study design appropriate for low level adult immigrant students 
learning another language and developing literacy in a second language. Efforts to design 
the study to match real-life conditions were overridden by an advisory group of high-level 
researchers with no experience in adult ESL programs. 


e Advisory panel members disagreed about what mattered in adult ESL literacy with some 
suggesting a phonics intervention while others supported a contextualized approach 
focused on English for work. There was no process for bringing together disparate points of 
view since it was not clear what needs and goals of the students in the programs would be. 


John Sabatini, Ph.D., Educational Testing Service 


Dr. Sabatini said he has been involved in national and international adult literacy 

John Sabatini, Ph.D., Educational Testing Service Dr. Sabatini said he has been involved in 
national and international adult literacy surveys as well as technology-driven programs to 
understand the components of literacy and foster professional development. He described the 
challenges of one large research effort funded by the Eunice Kennedy Shriver National Institute 
for Child Health and Human Development (NICHD), which sought to improve adult learners 
reading skills with research-based interventions. The interventions were informed based on 
findings from research demonstrating success among K-12 students. Dr. Sabatini and 
colleagues adapted the K-12 programs for use with adult learners in literacy programs. 
However, adult education involves much less class time and attrition rates are very high relative 
to K-12 education. In addition, as more literature was published, it became clear that the K-12 
evidence-based practices may result in more modest gains than originally expected. 
Nonetheless, Dr. Sabatini and colleagues had to continue with the programs as designed, to 
uphold the rigor of the study design. The published findings illustrate the barriers faced and 
some of the methodological approaches researchers took to accommodate the fact that nearly 
half of the adult students dropped out of the study. 


Noel Gregg, Ph.D., University of Georgia 


Dr. Gregg-presented research on the longitudinal analysis of select influences on adolescents’ 
with learning disabilities (LD) and emotional/behavioral disabilities (EBD) occupational 
aspirations. To inform the development of career theories and interventions to respond to this 
population, she and several researchers at The University of Georgia are undertaking research 
activities focused on adolescents with selected high incidence disabilities, which includes 
individuals in one of several disability groups commonly encountered in the classroom including 
learning disability (LD) and emotional and behavioral disorder (EBD). The proposed research is 
addressing the paucity of research on adolescents with selected high incidence disabilities (LD 
and EBD) by conducting three related strands of inquiry that, when completed, will provide (1) 
theoretical, (2) outcome-oriented, and (3) intervention/ applied perspectives of the transition 


3 Dr. Wrigley later noted that “[s]imilarly, since literacy development is part of any beginning ESL class, the control 
teachers were also likely to integrates phonics into their curriculum (there was no standard curriculum for students in 
the control group).” 
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from school to adult life for this student population. A brief summary of proposed research 
activities is offered next. Dr. Gregg is investigating (a) the purpose of investigating the 
association between educational outcomes and malleable factors (e.g., student attitudes, school 
location), and (b) examining the factors and conditions that may mediate or moderate the 
relations between malleable factors and education outcomes (e.g., gender, race, disability). 
Interestingly, Dr. Gregg said the models for evaluating risk and resilience among students with 
disabilities did not hold up during analysis of large databases, such as the Education 
Longitudinal Study of 2002 (ELS). Risk and resilience models operate very differently in work 
settings compared with academic settings. Dr. Gregg was skeptical they would hold up in adult 
education populations. She is currently comparing the effects of inclusion versus career- 
technical models on career aspirations among students with disabilities using the NELS, ELS, 
and the National Longitudinal Transition Study-2. 


Dr. Gregg discussed the Georgia and National Science Foundation-funded STEM (science, 
technology, engineering, and math) Accessibility Alliance (www.Georgiabreakthru.org ) to 
provide support to students with disabilities in STEM degrees and careers. The project involves 
virtual learning environments and social media tools effective for the secondary and 
postsecondary population with disabilities interested in STEM careers. Dr. Gregg suggested that 
researchers will need to provide greater attention to the type of research methodologies (e.g., 
social networking) that best allow for observation of learning across virtual and social 

networking tools.-She also discussed the importance of better understanding the factors 
influencing electronic mentoring with these populations. 


Finally, Dr. Gregg reported her evaluation activities on the EngageME — P.L.E.A.S.E. 

Innovation Fund (i3) grant awarded by the Department of Education: 2010-2015)which is 
developing a virtual personalized learning environment for middle and high school students. She 
discussed the importance of developing adult education personalized learning environments, 
particularly as a means to deal with the populations problems with time and space constraints. 


Daryl Mellard, Ph.D., University of Kansas 


Dr. Mellard said his research looks at similarities and differences between adults and children 
on reading and literacy acquisition. This is informative but has not yet translated to research 
instruments. Findings indicate that because of cognitive differences between adults and 
children, adults do not just need more drills and practice. Dr. Mellard added that motivation is 
extremely important. Motivation to enroll in a program must be distinguished from motivation to 
persist. 


Most adult education is provided through community-based organizations and community 
colleges, but the U.S. Department of Labor includes ABE and adult literacy programs in its 125 
Job Corps centers around the country. Dr. Mellard and colleagues aim to develop interventions 
to improve teaching at Job Corps centers, but the research faces several barriers. The number 
of programs and the sample of students from which to draw are much smaller than that of 
general adult education, and researchers must reach out across several States. The Job Corps 
centers offer GEDs, high school diplomas, and simultaneous training toward trade certifications 
(e.g., construction, health care). Many of the centers are privately operated and do not share 
information. The private operators compete for contracts every 5 years, so researchers focus on 
recent awardees that are likely to be in business for the duration of the study. 


The Job Corps centers allow open enrollment and new students with varying levels of 


preparedness are integrated into classes every week. Students range from those with very low 
literacy to those ready to complete their GEDs. On any given day, students spend part of their 
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time in academic classes and part in trade-based training. Finally, completion of the GED 
requirements can be used as a standard outcome measure across States, but every State has 
its own standards and requirements for trade certifications, making it difficult to compare 
outcomes on the basis of certification. 


Steve Reder, Ph.D., Portland State University 


Dr. Reder described what he learned from a 9-year study of 1,000 adults who represent the 
target population for ABE (but are not necessarily enrolled in classes). The study had 90 
percent retention over 9 years, but some subjects dropped out because of competing 
priorities—busy lives, migration, etc. Dr. Reder emphasized that we need to disentangle study 
subject attrition from program attrition. The former is a methodological factor, while the latter is a 
programmatic factor. Understanding attrition is particularly important for education that is not 
compulsory. 


In terms of research, the “dosage” of adult education cannot be controlled by the researcher, so 
it is difficult for a researcher to determine whether comparisons are accurate. Another thing that 
complicated his longitudinal study was that some measures used for short-term testing were not 
always stable over time because the study lasted so long. Scales that work well for 
psychometrics often are not stable over time, said Dr. Reder. 


Dr. Reder said the field is not using an appropriate logic model to evaluate the impact of 
programs. In trying to compare adult education participants with nonparticipants the impact 
depends on the measures used and the time scale. Programs do have short-term, direct 
impacts on some measures, such as literacy and numeracy practices, but proficiency measures 
usually require a longer time to demonstrate an effect. 


Moreover, a learner-centered lens would reveal different aspects than one that is program- 
centered, said Dr. Reder. By looking more broadly at participation (and including, for example, 
self-learning and expanding the measures to follow individuals over time regardless of 
continued program participation), we would get a better picture of how adult education programs 
work. Dr. Reder called on methodological experts to help improve the logic model that underlies 
how we look at programs. 


Daphne Greenberg, Ph.D., Georgia State University 


Dr. Greenberg identified some of the challenges she and her colleagues faced as they tested 
several reading interventions among adults reading at the third to fifth grade levels. First, the 
use of grade levels designed for children to assess the skills of adults is problematic. For 
example, adults may demonstrate different strengths and weaknesses than children reading at 
the same grade level. The significant attrition rates in adult programs brings into question how 
many hours of instruction a given student actually receive. Dr. Greenberg’s study found that 
despite teachers’ high rates of fidelity to the various interventions and enthusiasm among 
teachers and students for their perceived gains, none of the interventions yielded very strong 
effects—as demonstrated by reading- related assessments. 


Measuring the impact of instruction is complicated by the lack of psychometric research in the 
reliability and validity of commonly used reading-related assessments for adults who struggle 
with very basic reading skills. Also, dosage varies—students may receive the same number of 
hours of instruction, but one may attend sporadically over several months, while another may 
attend regularly for several weeks. Thus, the hours of instruction can have different 
characteristics. Dr. Greenberg wondered how to capture that variation in analysis. Dr. Wrigley 
pointed out that in adult ESL classes, those students who attended class for a concentrated 
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time made more progress than those who attended for the same number of hours over a longer 
time. 


Discussion: Program- and System-Level Characteristics That Affect Research 

Dr. Sabatini pointed out that children get 12 years of immersion education through daily learning 
that includes reading, writing, and homework in all subjects. When they go on to college, they 
spend another 4 years reading and writing, and the outcomes are not measured until they 
complete those 4 years. By contrast, adult education involves 50-100 hours of instruction 
outside of other commitments. Furthermore, Dr. Sabatini said, we evaluate all the adult 
education participants together as a group, without distinguishing those who do very well right 
away from those who do not move forward (and whose failure to progress probably could have 
been predicted with testing). 


Dr. Sabatini asked how we could follow individuals over time to demonstrate the impact of 
programs. He suggested that it is as important to invest in understanding real-world impact-such 
as jobs-then in evaluating learning gains. 


He also noted that in some cases, more money is spent on evaluation than on the programs 
themselves. Dr. Wrigley agreed, saying that disproportionate spending (millions on research vs. 
perceived limited funds for local program improvement) leads to resentment among many 
teachers and some coordinators, teachers report being tired of putting in time to meet the needs 
of researchers, and many feel they should at least get something in return, such as materials 
they can use in the classroom or other tools that would make their teaching lives easier, rather 
than more burdensome, Teachers often feel that the final reports of studies don’t speak to them. 


To better measure impact, Dr. Sabatini suggested understanding individual goals for enrolling in 
programs and aligning outcome measures with those goals. Dr. Reder agreed, noting that 
impact develops over time, noting that individuals don’t complete a program and then 
immediately get a better job or higher income. Literacy grows over time as well, said Dr. 
Sabatini, as students read on their own more and more. Thus, it may make sense to test a 
participant’s reading skills some time after the end of the program. 


Dr. Gregg pointed out that participants’ aspirations are influenced by a lot of things well before 
they ever attend a literacy class, including family literacy and the family’s aspirations for the 
child. Dr. Hedges agreed with Dr. Reder’s contention that the underlying logic of program 
assessment is flawed. Looking at an individual over a lifetime shows that outcomes are the 
result of several interventions and experiences, not the impact of a single program. Dr. 
Greenberg also agreed, noting that participants drop in and out of programs over time. 


Ms. Serino said that Massachusetts developed its own assessment, the Massachusetts Adult 
Proficiency (MAP) test, which is grounded in the State’s own framework for adult education. The 
test provides feedback to teachers about student performance. Ms. Serino said assessment 
tests should be closely aligned with program content. Dr. Hedges pointed out that most adult 
assessments are adapted from tests designed for younger people and that the tests are 
possibly targeted at the wrong level to be informative or the test items do not relate to adults in 
the same way they do for children. 


Dr. Sabatini said assessments can be developed and refined to be more sensitive to outcomes 
in adults at different levels. However, even good assessments do not demonstrate significantly 
better outcomes. Accumulation of learning over time does yield good outcomes. The question is 
whether adult education efforts trigger learning mechanisms that help participants improve their 
literacy over time—and we do not measure that, said Dr. Sabatini. 
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Dr. Reder suggested considering how the system should change to allow for a broader range of 
programs and new models that use multiple learning modalities and evaluations over longer 
time spans. The system should stimulate and support innovation, he said. Dr. Sabatini agreed 
that we should encourage innovation and avoid punishing programs for taking risks. Dr. Reder 
and Ms. Serino agreed that program directors are very wary of taking risks that might affect their 
funding. 


Discussion: Teacher Characteristics That Affect Research 

Dr. Finnigan pointed out that K-12 teachers receive a lot of instruction and support, while adult 
education teachers often have no training. Understanding variability in teachers’ background, 
educational experiences, and supports as adult educators may be useful. Dr. Gregg said it may 
be time to reframe teachers into mentors. Technology can now deliver information, taking on the 
role that teachers used to play. 


Dr. Wrigley noted that working conditions for adult education teachers are very poor—they often 
have short-term contracts with few benefits. Teacher turnover is high (the average is 3 years), 
which hinders the ability to build program capacity with sound teaching. Teachers often have a 
college degree—but not in the field they teach—and they may have teaching experience—but 
often no or little experience teaching adults or low-level learners immigrant learners 


Currently, professional development usually involves going to workshops.* New teachers may 
spend 10-20 hours a year on development, while experienced teachers spend even less. 
Sustained professional models that include face-to-face training plus virtual follow-up, 
interaction, self-reflection, and self-determined interventions seem to build greater awareness 
than workshops, said Dr. Wrigley. Dr. Gregg added that social networking can help us 
understand the connections among teachers, learners, administrators, policymakers, and 
others. 


Discussion: Student Characteristics That Affect Research 

Dr. Mellard noted that some surveys ask students to identify their goals for attending adult 
education. Students usually state a primary goal (e.g., job, citizenship), but 40 percent mark 
“other” as their goal. In evaluating program impact, Dr. Mellard said, it is important to capture 
the secondary goal(s) too, because it may affect a student’s motivation and persistence. 


Dr. Greenberg said her research found a number of factors related to persistence, such as the 
type of program, age of the student, the student’s socioeconomic status (SES), and interest in 
or avoidance of reading. Although these factors seem logical, it’s not clear why these aspects 
rose in relevance above others. Dr. Greenberg suggested looking beyond a participant’s 
reading level to the demographic and other factors, then customizing programs to the students. 


Dr. Wrigley pointed out that the characteristics of ESL students are much different from those of 
other adult learners. The range of backgrounds is much broader. For example, some ESL 
students earned college degrees in their native countries, but others (29 percent of all 
immigrants and 68 percent of Mexican immigrants) do not have a high school education. Dr. 
Wrigley added that without literacy in one’s native language, it is a difficult and lengthy process 
to become literate in English. She suggested that established programs partner with other 


Di Wrigley later noted that “[rlesearch suggests that on short workshops do not result in changes in practice.” 
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organizations that provide literacy education in the student's first language. She stated that we 
need to stop thinking about ESL as ABE with an accent.° 


Dr. Gregg said SES is not always holding up as a predictor of success. In evaluating risk and 
resilience, however, SES is an indicator of aspirations. Dr. Dowd pointed out that teachers have 
their own cultural biases and that it may be helpful to make teachers more aware of racial/ethnic 
or cultural considerations that affect how they teach. African Americans are overrepresented in 
adult education, said Dr. Dowd, and we should not assume that teaching practices are always 
neutral or positive when they could in fact have negative effects. She suggested studying 
teacher practices as well as student goals. Dr. Dowd also hoped some data could focus on 
measuring how participants take what they learn and use it in their own communities, not just in 
academic discourse. 


Dr. Wrigley concurred with Dr. Dowd’s observations about bias. Bilingual classes often have 
wide disparities in SES, even among people from the same country. Dr. Wrigley said that in 
some cases, the more affluent or better educated may correct the Spanish of the less 
affluent/educated, effectively silencing them. Teachers must work to build collaboration, she 
said. Furthermore, more affluent and better educated students can easily hijack the class and 
sway the teacher’s attention toward their needs. In both cases, low educated immigrants — 
those with the greatest need for support — lose out. 


Recap: Potential Areas for Exploration 

Following lunch, Dr. Larson and Dr. O’Donnell summarized some of the areas of exploration 
discussed throughout the morning. (Appendix A represents a final summary of topics and 
questions for exploration.) The following comments were made: 


e Knowledge gained from research on college students may be a resource for understanding 
adult experiences in education. 


e Assessments should be specific to the aspect of literacy measured (e.g., language, literacy, 
decoding, comprehension). 

e Assessments should follow students over time. Instead of measuring literacy, long-term 
assessments should evaluate quality of life, aspirations, and self-image, for example. 

e Expectations for the program impact and individual performance should align with the 
research question and the methods used. 

e Frame the research around individuals, not programs, and tailor the methods accordingly. 

e Assess for increases in learning, aspiration, interest, etc., beyond the program. 


e Consider the potential of technology-based learning and its ideal applications. 


Methodology Perspectives 
Elizabeth Stuart, Ph.D., Johns Hopkins University 


Dr. Stuart pointed out that researchers in other fields face similar challenges and thus may have 
solutions. For example, persistence is analogous to the issue of maintaining a patient on 
antipsychotic medications over time, and length of time on the drug is an important outcome. 
The need to ensure that program outcomes are relevant to the participant is similar to the new 
focus in medical research on patient-centered outcomes, such as function and quality of life 


° Dr. Wrigley later noted that “[p]rocesses involved in Second language acquisition and second language literacy 
development are quite different from the processes by which native speakers of a language develop their literacy 
skills.” 
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over test results. Prevention science focuses on the long-term impact of an intervention that 
may not show significant short-term gains. Finally, Head Start programs may be a laboratory for 
research on professional development. 


Dr. Stuart asked whether State and national data could be combined to answer broad 
questions. She also explained that propensity scores are useful for comparing two groups when 
random assignment is not feasible, such as adults who self-selected into different programs. 
Propensity scoring may be able to distinguish the characteristics of people who drop out 
compared to people who persist in programs. To work well, however, researchers need good 
predictors and good covariates. 


Dr. Wong noted that pre-tests are useful for matching populations and multiple pre-tests are 
even more effective. Dr. Sabatini noted that with multiple tests, adults may start to apply 
strategies for answering questions rather than demonstrating knowledge. Dr. Wrigley agreed 
that adults are not motivated to engage with and do well on tests but rather just focus on 
finishing them. Dr. Stuart said the possibility of linking K-12 test data with findings on the same 
students as they become adults is exciting. 


Kara Finnigan, Ph.D., University of Rochester 


Dr. Finnigan described her efforts to evaluate relationships and connectedness through social 
network analysis. For example, such research may look at mentors of adult learners and the 
influence of resources that flow between the mentor and student, e.g., advice. Social network 
analysis can identify which connections (e.g., among learners and teachers) have the most 
impact. Studies using social network analysis in public health, for example, have identified the 
“brokers” of information. This is important because these brokers can then serve as a point of 
access to the other people in the network. It can also demonstrate where there are opportunities 
to provide support (e.g., if people are isolated from certain resources, such as academic or 
emotional support). This approach may be useful in adult education research. 


Dr. Finnigan cautioned that adult learners are harder to analyze than their teachers, especially 
when literacy challenges are involved, because you have to ask questions in a way that focuses 
on complex connections. Student attrition rates, as well as rapid teacher turnover, can make it 
difficult to use social network analysis to evaluate changes in connectedness over time between 
adult learners and their teachers. Currently, a lot of existing research from social network 
analysis evaluates the importance of relationships in terms of both individual and organizational 
performance. This type of examination of the underlying relationships of adult learners may be 
useful in looking at persistence and attrition. Dr. Finnigan suggested that examination of the K- 
12 literature on dropouts may also inform research on the persistence and attrition of adult 
learners. 


Dr. Greenberg pointed out that some anecdotal reports indicate that attendance improves when 
class participants are responsible for each other’s’ attendance (e.g., finding out why a student 
did not attend and encouraging that student to return). Dr. Mellard noted that groups that have 
common interests are more likely to stay together and that classrooms can build a social 
network. Dr. Finnigan pointed out that social network analysis is ideal for modeling isolation and 
connectedness. Researchers can use it to find points where students connect with each other or 
teachers and the strength and quality of these relationships. Dr. Wrigley said that there are early 
warning signals (i.e., red flags to be paid attention to). For example, a lack of participation and 
engagement in the classroom indicates that a student is at high-risk of dropping out. She added 
that in ESL classes, participants tend to go to the same programs as their friends, regardless of 
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whether it is the best program for them, and this illustrates how important peer support is to 
engagement. 


Vivian Wong, Ph.D., Northwestern University 


Dr. Wong said methodologists have done a good job over the past 10 years developing 
methodologies to address causal questions but have not done as well integrating these causal 
questions into a more general framework for program evaluation, so more work needs to be 
done on that frontier. She noted that selection bias is still a frequent concern among adult 
education researchers. In the past, researchers have addressed concerns about selection bias 
through random assignment, but the sample size in these studies were small and did not 
generalize well to other populations of interest. In addition, the studies suffered from significant 
attrition problems. Dr. Wong suggested that randomized control trails in the job training literature 
from the 1980s and 1990s might be models for how adult education researchers can evaluate 
their own programs in field settings. She noted that much of what we know now about what 
goes wrong in field research (e.g. treatment noncompliance) and how to address these 
problems come from job training studies. In addition, researchers should consider including 
design options that directly address potential challenges for field evaluations. For example, 
researchers could imbed randomized experiments in contexts where there are long waiting lists 
to participate in a program or receive a service. If attrition is a problem, then researchers could 
consider randomizing incentives for participation in an adult education program, such as 
providing free child care, stipends, or transportation to the program. 


A regression discontinuity design can be used when participants are assigned into a program on 
the basis of their score on some continuous assignment measure and a cutoff threshold. 
Individuals with scores above the cutoff may be assigned into the treatment condition while 
those with scores below the cutoff do not receive treatment. Treatment effects are measured by 
the size of the discontinuity in the regression line that occurs exactly at the cutoff. One possible 
application of the RD design in adult education settings is when grants are awarded to 
recipients on the basis of their application score and a cutoff threshold. The advantage of the 
RD design is that it allows program officials to assign participants or organizations whom they 
feel are deemed most “in need” or “worthy” of receiving treatment. The interrupted time series is 
another approach to consider. For example, if longitudinal records of student performance are 
available, researchers could compare student test results before and after the introduction of an 
adult education program to evaluate outcomes of interest. 


Dr. Wong said she and her colleagues believe that comparing non-equivalent education sites 
and participants (matching) works better when the units come from similar populations—for 
example, within a State rather than across States. Attrition is probably a useful outcome in 
understanding the covariates and factors that predict selection and persistence, Dr. Wong 
noted. Randomly assigning students to a usual-procedures groups or one that receives a lot of 
incentives may be useful in looking at the effect of persistence itself. Finally, researchers should 
consider a design option where individuals or units are randomly assigned into either a “random 
assignment” arm or an “observational study” arm. In the random assignment arm, units are 
randomly assigned to receive the treatment or not. In the observational arm, units are allowed to 
select into the treatment condition. Using rich covariate information from the observational arm, 
researchers can then apply matching methods to estimate their quasi-experimental treatment 
effects, and compare these effect estimates to the gold standard randomized experiment 
results. If the quasi-experimental methods are able to replicate the experimental benchmark 
results, then the researcher has some reassurance that they have gathered the appropriate 
covariates for modeling the selection process or outcome in the quasi-experiment to yield 
unbiased effects. 
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Dr. Stuart noted that one option for evaluating attrition is to invest in the follow-up of a small, 
randomized sample. Participants briefly debated whether providing incentives, particularly 
money, is effective. Dr. Stuart noted that incentives can be used to learn whether persistence 
matters. 


Alicia C. Dowd, Ph.D., University of Southern California 


Dr. Dowd explained that her approach, “symposium research,” involves combining traditional 
methods of quantitative and qualitative research with action research—the practice of 
developing tools for practice that are based on the experience of practitioners. In the same way 
that Pennsylvania’s professional development asks that teachers reflect on their classroom 
experiences, Dr. Dowd and colleagues ask that educators review data on themselves. 
Teachers’ experiences and their input on what works are combined with other research findings, 
and, consequently, the research focus shifts to the activity setting of teaching and learning 
rather than to being solely on learners’ outcomes. Dr. Dowd said the activity settings of 
teaching and learning should be a unit of analysis, and data should be collected on teachers as 
well as students. She emphasized that research should not consist solely of outside entities 
looking in but, rather, should involve collaboration among researchers and practitioners to 
understand how the practitioners produce knowledge. 


As mentioned earlier, teachers may be more like mentors or coaches now and learning may not 
necessarily happen in the classroom. The activity setting goes beyond the immediate delivery of 
information. As an example, Dr. Dowd briefly described efforts to use symposium research to 
overhaul the education of first-year science students at a polytechnic university to take into 
account concepts such as the intent of programs, the structures involved, the materials used, 
the effects of teacher practices on learners with different backgrounds, and the integration of 
research into the activity setting. The curriculum involves service learning, community 
engagement, peer groups, and other activities that go well outside the boundaries of school. Dr. 
Dowd pointed out that symposium research is a mechanism for robust research in a highly 
dynamic, real-world setting. 


Dr. Wrigley said that much attention is focused on evaluating interventions but little is given to 
understanding how to build an intervention that makes a significant difference, such as how to 
get buy-in and enthusiasm from teachers and students so that classroom dynamics are 
improved and students are likely to stay longer, resulting in greater gains. Dr. Dowd added that 
the educator’s perception of efficacy and control of the teaching environment also plays a role. 
Researchers also have an ethical obligation to identify interventions that have not worked or 
are not likely to work given past results® and discourage spending on such efforts, said Dr. 
Wrigley. Ultimately, said Dr. Dowd, the symposium approach is not very different from current 
approaches but focuses as much on cognitive and behavioral changes among learners and 
teachers as on outcomes. Dr. Dowd later said that it also requires greater collaboration among 
researchers using different data sources and analytical techniques. This entails a willingness 
(and patience/openness) to integrate across different research paradigms to produce 
knowledge. 


Larry Hedges, Ph.D., Northwestern University 


Dr. Hedges observed that the discussion seemed to be raising two different strategies for 
moving forward: conventional research on existing programs on one hand and a radical 


° Dr. Wrigley later noted that “[mJost of the recent experimental studies in ABE amd Adult ESL have not shown 
significant effects.” 
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restructuring of both programs and research approaches on the other. Taking the conventional 
route, we can improve the results by paying attention to the proportionality between cause and 
effect—placing realistic expectations on outcomes in proportion to the resources invested. Small 
investments in financial products usually result in small gains. Only in social services and 
education do we think, “We can invest nothing and get enormous results,” said Dr. Hedges. 
Along the same lines, he noted that we must consider that 100 hours of education is equivalent 
to 1 month of education at the elementary school level and measure effects accordingly. 


Conventional research approaches could be used to encourage persistence, Dr. Hedges noted, 
and to identify the predictors of enrollment and persistence. For example, if persistence is 
thought to be crucial to obtaining impact on other variables, research on factors to improve 
persistence in programs is very worthwhile. Understanding how to model persistence as a 
function of other factors that can be measured would be valuable in mitigating the expected 
attrition and missing data. 


Alternatively, researchers could pursue the person-centric model suggested by Dr. Reder and 
evaluate the effects of programs over time as an individual pursues his or her goals. In addition, 
researchers could reconsider whether the outcome measures used to date are relevant. A 
person-centric model would focus on the goals and outcomes identified by individuals and 
would be measured by that individual's perception of whether those goals were achieved. 
Whether we pursue conventional or alternative research approaches, said Dr. Hedges, we 
should seriously question whether we are serving ourselves or our clients. 


Discussion 

Dr. Stuart pointed out that research is usually funded for 5 years at the most, so investigators 
often do not have the opportunity to identify long-term effects. Given the discussion at the 
meeting Dr. Stuart believes that an important area for further research and development is on 
measures and how to measure adult literacy and the relevant outcomes. Dr. Hedges agreed, 
saying research indicates we know less about measuring adult literacy than we thought we did. 
The current assessment tools are a hybrid of measures for children and adults, said Dr. 
Sabatini. 


Dr. Wong said that if researchers believe an intervention’s effects only manifest in the long-term, 
they should build a strong case to support that belief based on observed patterns (rather than 
fishing for findings 5 years later). Dr. Hedges said a person-centric research approach could 
include some reasonable theories on long-term effects. 


Dr. Wrigley pointed out that motivation evolves over time. She gave the example of an ESL 
digital literacy class. Several people who were only seeking an ESL class found they were very 
motivated by the digital component and their goals changed as a result. 


Also, people want to learn more than the basics of reading and writing and researchers should 
consider the participants’ other goals. Dr. Dowd said the book Stuck in the Shallow End: 
Education, Race, and Computing makes a good argument for the need to understand how goals 
are shaped, how malleable they are, and how educators play a role in shaping them. Dr. Dowd 
later noted that overall the discussion at the meeting was “deracinated;” seldom addressing the 
racial or ethnic characteristics of the learners or teachers, or the social status of either in their 
different communities. She noted that there were a few exceptions during which panelists 
commented on race and ethnicity as important elements to the discussion, such as Dr. Wrigley’s 
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points about ESL students.’ She contended that researchers won't be able to fully characterize 
the problems of adult literacy and ABE, or conduct research to address them, in the absence of 
an explicit focus on issues of race and racial discrimination in U.S. society. 


Key Issues for State Programs 
Dr. Larson invited participants to describe the most pressing issues. Ms. Serino said State 
programs would benefit from paying more attention to the following: 


e Effectiveness of new professional development models, such as intensive cohort- or 
classroom-based models 


e Rationale for investment in adult education 
e Effective [instructional] strategies for adult learners 


e Relative effectiveness of community colleges compared with other organizations that 
provide ABE 


e Mechanisms to support independent innovation 
e Using technology to support adult learners in developing literacy skills 


Mr. Westover said he struggles with where to begin addressing the needs of poor Latino 
immigrants in Pennsylvania who lack education and literacy in both their native language and 
English. Manufacturers in the State have jobs for this population, but the immigrants need better 
communication skills. Community colleges in Pennsylvania have expressed interest in 
collaborating to address the issue. Dr. Dowd suggested exploring teaching methods that tap into 
an individual’s fund of knowledge (a.k.a. “asset-based” instruction) to reach immigrants with 
poor education and communication skills. Such an approach focuses on individual skills for 
success (e.g., critical thinking and survival skills). Dr. Dowd added later that other researchers 
are giving attention to the concepts of funds of knowledge in marginalized communities (e.g. 
Luis Moll, Robert Rueda, Ebony McGee) and to culturally responsive pedagogy or critical 
pedagogy (e.g. Geneva Gay, Gloria Ladson Billings, Kris Gutierrez). It would be important in 
future meetings to incorporate their research. A more racially and ethnically diverse participant 
group might also expand the range of perspectives available for problem framing and research 
design. 


Also, Mr. Westover said Pennsylvania adult education data could be combined with data from 
the State departments of labor and welfare to track participant outcomes over time. More work 
should be done to identify strategies helping people with learning disabilities. 


Areas for Exploration 

Dr. Larson asked participants to delve more deeply into questions to be addressed and the tools 
needed to address them. The complete list of topics and questions for exploration appears in 
Appendix A. Participants offered the following suggestions: 


e Build capacity and increase investment so that results are proportional to investment. 
e Prioritize research questions. 
e Evaluate innovative programs and the keys to successful implementation (e.g., 


Washington’s Integrated Basic Education and Skills Training [I-BEST] program, which uses 


’ Dr. Dowd later also noted Mr. Westover's comments on poor Latino immigrants’ needs as another example of the 
importance of incorporating social factors into the conversation. 
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a team-teaching approach and Texas’ program for providing contextualized ESL and GED 
education). 


e Study the impact of dropping in and out of programs over time. 


e Determine whether an individual’s progression to higher education levels (according to the 
NRS) links to measurable social and economic benefits. 


e Develop methodology to support secondary analysis of data on the relationship between 
hours of attendance and educational gains. 


e Determine what State and national data are available for in-depth analysis. 


e Take advantage of natural experiments, such as the upcoming transition to new GED 
testing. 


e Evaluate the impact of the Spanish GED with and without English language skills training. 


Participants noted that a new logic model would require a better understanding of the current 
logic model underpinning adult education research, common terminology (e.g., definition of 
persistence), and identification of appropriate performance expectations. Dr. Wrigley said it 
would be helpful to have a summary describing gaps in the current system of adult education — 
what services seem important but are not being provided, at least not on a large scale (e.g., 
programs that integrate job training with English for specific purposes). Dr. Dowd said a new 
logic model would allow researchers to get at the “why’—the value of adult education—by 
enabling them to determine impacts on such outcomes as healthy communities, civic 
participation, and family involvement in children’s lives. 


IES Support: Allen Ruby, Ph.D., Associate Commissioner, Policy and Systems 
Division, NCER 

Dr. Ruby described IES funding opportunities that investigators could pursue. He encouraged 
researchers to call him with ideas so that he could direct them to the right program officer. 


Research and Career Development Funding Opportunities 

Participants discussed the value of partnering across institutions to design multidisciplinary 
research protocols and Dr. Ruby encouraged participants to consider applying for a 
Researcher-Practitioner Partnerships in Education Research grant to support such an effort. He 
also encouraged participants to propose research on the new GED test and to seek funding for 
training postdoctoral students in adult education research. 


Dr. Albro pointed out that the Early Career Development and Mentoring Program in the National 
Center for Special Education Research (NCSER) can also support adult education researchers. 
Ms. McLaughlin, Deputy Commissioner of NCSER, added that NCSER funds a research 


program on Transition Outcomes for Secondary Students with Disabilities. 


Participants suggested that NCER include more experts in the field of adult education on grant 
review panels and educate other reviewers about the unique challenges adult education faces. 
Dr. Stuart recommended that applicants better articulate the challenges in their applications. 


Conclusion and Adjournment 

Dr. Larson thanked the participants for their hard work. She plans to provide participants with a 
summary of the meeting results and ask for input on prioritizing the next steps. 

Dr. Larson adjourned the meeting at approximately 4:30 p.m. 
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Appendix A: Topics and Questions for Exploration 


Evaluation and Effectiveness 


Research Questions 

What are the most effective strategies for supporting adult learners? 

What innovations show promise? 

What are the factors in building a successful intervention that makes a difference (such 
as teacher and student buy-in)? 

What is the relative effectiveness of community colleges compared with other 
organizations that provide adult basic education? 

Do learners benefit from multimodal, multidimensional programs? 


Related Topics: Assessment/Measurement Tools 

e Appropriateness for adult learners 

e Need for strong measurement techniques or theoretical basis for adaptation to adult 
learners 
Measurement of the right construct 
Specificity to the outcome of interest (e.g., language, literacy, decoding, 
comprehension) 

e Measurement over time/long-term effects of education on individual quality of life, 
aspirations, and self-image, as well as concrete measures such as getting a job ora 
raise or improved literacy 

e Accounting for actual class time and high attrition rates among adult learners 

e Accounting for variability in the background, experiences, and supports of teachers 
of adult learners 

e Measurement of self-determination 

e Distinction between motivation to enroll and motivation to persist 

e Alignment of outcome measures with individual student goals (primary and 
secondary) 

e Demographic and other factors beyond reading level 

e Measurement of evolution of motivation over time 

e Realistic expectations of outcomes in proportion to the resources invested 


Related Topics: Data and Databases 

e Availability and quantity of data 
Quality and accuracy of available data 
Comparability across data systems 
Identification of covariates 
Determination of outcomes of interest 
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Student-Centered Issues 


Research Questions 

What strategies are effective for helping adult learners with learning disabilities? 
What critical thinking skills do adult learners bring that will help them succeed in adult 
education? 

What factors are associated with persistence? 

What is the relationship between dose (hours of attendance) and measured gain? 
What is the relationship between measured gains in literacy and social and economic 
advancement? 

What are the social impacts and implications of failing to pass literacy tests? 

Can adult education improve performance on individual measures of learning? 

How can programs address the needs of English as a second language (ESL) students 
with very low native literacy skills and poor communication skills? 


Related Topics: Research Challenges 

e Application of K-12 research to adult learning 

e Identifying the right population for comparison/extrapolation (college 
undergraduates, high school students) 

Translation of research into digestible, portable, accessible materials 

Usefulness of research for planning and decision-making 

Development of models for adults 

Challenges of using social network analysis with adult learners given literacy and 
second language issues 


Related Topics: Setting 

e Setting characteristics 

e Asset-based instruction/acknowledgment of participants’ life skills 

e Setting-specific program components versus broadly applicable components 


Related Topics: Dosage 

e Defining and controlling dosage 

e Distinction between subject attrition and program attrition (methodological versus 
programmatic factors) 

e Feasibility of interventions/programs (student time, teacher skills, innovative versus 
conventional) 


Long-Term Impact/Person-Centered Model 


Research Question 
What are the long-term effects of program participation? 


Related Topics: New Logic Model 

e Realistic expectations for impact (familial, societal) 

e Long-term measures to capture learning experiences before, during, and after the 
program 

e Measurement of advances in aspirations and self-determination 
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e Ideal models: busy intersection (learner is not in the program for long, but it is 
important where they go next) versus parking lot (the longer the student stays, the 
better) 

Defining and conceptualizing a program 

Incorporation of multidimensional, multimodal components 

Student characteristics that interact with the learning process (e.g., age, 
socioeconomic status, language, previous education experience, race, ethnicity, 
culture, persistence, motivation) 

Interaction of student, teacher, and program characteristics 

Need for common terminology 

Definition of growth/evaluation of long-term trajectory 

Individual versus program focus (person-centered approach) 


Innovation 


Research Questions 
What impact do innovative programs have on the success of adult learners? 
What are the keys to successfully implementing innovative programs? 


Related Topics: Support 

e Funding for research (sustainability, long-term funding, political support) 
Research resources 

Discrepancies in research funding across States 

Rationale for investment in adult education 

Mechanisms to support independent innovation 


Professional Development and Instructor-Related Issues 


Research Question 
How effective are new professional development models, such as intensive cohort- or 
classroom-based models, on the quality of instruction? 


Related Topics: Instructor Factors 

e Implementation/fidelity 

e Characteristics and skills of the instructor (including cultural biases) 
e Compensation tied to the quality of instruction 


Technology 


Research Question 
What technology can be used to support adult learners and increase literacy? 


Related Topics: Technology 

e Ideal combination of virtual and face-to-face learning 

e Ideal characteristics of a web-based program 

e Methodology to understand how students use virtual models and social media 
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Potential Research Tools and Designs 


Propensity score matching 

Random assignment and randomized controlled trials (RCTs) 

Action research—collaboratively developing and validating tools for practice that are 
based on the real-world experience of practitioners 

Regression discontinuity 

Interrupted time series 

Social network analysis 

Survival analysis 

Follow-up on a small, randomized sample of participants 

Incentives to persist in programs 

Combining longitudinal education data with data on labor and welfare/human 
services 

Natural experiments, such as those provided by the impending GED (general 
education diploma/general equivalency diploma) test changes (e.g., exploring the 
major variation among the States) and by the Pell Grant policy changes 

Impact of the Spanish GED with and without English language skills training 


Steps Toward Advancing Adult Education Through Research 


Develop multidisciplinary teams. 

Include adult learner experts on grant review panels. 

Build the capacity of the field to conduct research in this area. 

Facilitate connections among States that have the capacity for collaboration. Identify 
States with compatible State unique identifiers, jobs data, linked teacher and student 
data, etc. 

Generate a new logic model. 

Allow for other opportunities to expand research beyond the causal model. 
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